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ABSTRACT 


Deblurring capabilities would significantly improve the Flight Science Support 
Office’s ability to monitor the effects of lift-off on the shuttle and landing on the orbiter. 
This summer a deblurring program was written and implemented to extract information 
from blurred images containing a straight line or edge and to use that information to 
deblur the image. The program was successfully applied to an image blurred by 
improper focussing and two blumed by different amounts of motion blurring. In all cases 
the reconstructed modulation transfer function not only had the same zero contours as 
the Fourier transform of the blurred image but the associated point spread function also 
had structure not easily described by simple parameterizations. The difficulties posed by 
the presence of noise in the blurred image necessitated special consideration. An 
amplitude modification technique was developed for the zero contours of the modulation 
transfer function at low to moderate frequencies and a smooth filter was used to 
suppress high frequency noise. 
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I. Introduction 


Photographs of lift-off and landing of the shuttle are taken by the Flight Science 
Support Office. These are used to provide immediate identification of any potential 
damage to the orbiter as well as to identify sources and effects of any debris created 
during these crucial periods. The Space Shuttle Earth Observation Office is the 
repository of photographs taken by astronauts of environmental, geological, 
meteorological and oceanographic phenomena observed in orbit Inadvertently many 
photographs are blurred by improper focusing, relative camera-object motion etc. To 
gain access to important information contained in such photographs, it is necessary to 
apply image processing techniques. Tlic more sophisticated of these involve digitization 
of the raw images and subsequent computer processing of the digitized images or their 
Fourier transforms. 

The research this summer involved developing and implementing a program that 
would extract information on the point spread function (psf) directly from any blurred 
image containing a straight line or edge. Although the technique would work for more 
general blumngs it was especially useful for the two most common types of blurring: 
improper focusing and motion blurring. In the case of motion blurring the intensity at 
a point is spread along a line. Although it need not be uniform along the line, for 
simplicity it was assumed to be along a straight line. In tire case of improper focusing 
the intensity of a point is spread to neighboring points with the relative intensity 
depending only on the distance from the initial point The distribution of intensities of 
each point in the "sharp" image to the points in the blurred image is given by a 
convolution sum of the "sharp"image and the point spread function. The advantage of 
a Fourier Transformation (FT) is that it reduces this sum to a product If there were no 
noise, the FT of the desired image could be obtained by simply dividing that of the 
blurred image by that of the psf. However, for the cases of interest the Fourier 
transform of the psf (MTF) has contours of zero amplitude and becomes small at high 
frequency. Consequently noise in frequency regions where the MTF is small would 
result in serious degradation of the signal to noise ratio. Two techniques were applied 
to minimize such degradation. The first is the implementation of a two parameter high 
frequency filter. The second is the imposition of a maximum to minimum ratio limit for 
the amplitude of the filter. This is important at low to moderate frequencies. There are 
many excellent tests dealing with digital processing (1,2,3) and discrete Fourier 
transforms (4,5) that provide background for the work described in this report 


II General Approach and Theory 


HI Convolution of the Point Spread Function and Noise 

In general a blurred image is the result of light that would have arrived at a 
particular point being spread over a number of points. If the distribution is the same for 
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all points in the image and if the light is incoherent then in the absence of noise the 
final intensity at a point (pixel) located by m = [mji] would be given by 


(la) 


fi - 5 «• 


where p is the point spread function, i.e. the image of an ideal point source in the object 
plane, is the sharp image that would have formed in the absence of blurring and 
noise. 

For the images of interest the digitization process yields an image made up of 
NxN pixels (N=512). due to the blurring bringing light rays into the image region that 
they would have otherwise not have entered it, the summation in this expression should 
include m’s outside the range of 0 to N-l. Nevertheless, the sum is usually assumed to 
be limited to this range to avoid having to deal with an "underdetermined problem"(6). 

Noise is always present in the image. It can be the result of electronics, 
inhomogeneous chemical processes in the original film, etc. Noise can be divided into 
two contributions: one independent and the other dependent on the signal, i.e. f (7). In 
the latter case it could be incorporated into the point spread function. Since we have 
little knowledge of the nature of the noise, except that it is usually dominant at high 
frequencies, we will describe it by an unknown function n,, and write the expression for 
the blurred image as 

(lb) £ ~ + n a 


The next step is to invert the sum in equation 1 and solve for f. Since the 
summation is usually referred to as convolution, the inverse process is referred to as 
deconvolution. The basic technique is to expand all functions in terms of a complete set 
of basis functions which will turn the summation into a simple product of the expansion 
coefficients. 

D.2 Discrete Fourier Transormation 


For simplicity we use the traditional discrete Fourier expansion 


(2a) 


f* * ^ F t 


where z = exp(i2;t'N).The inverse Fourier transformation is then given by 

* * “ 
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In the following we will use lower case letters, e.g. f, for the spatial images and 
upper case letters, e.g. F, for the Fourier transformed image. The Fourier transform of 
the point function, psf, is called the modulation transfer function, MTF. 

In terms of the Fourier tranformed function Eq. 1 takes the form: 

(3a) Ft b = PfFr+N f 

Deleting the common subscripts and solving this equation for the Fourier transform of 
the sharp image function yields 

(3b) F* = F h fP - NIP 


If the modulation transfer function and the noise, N, were both known then 
equation 3b could be used to find P and a subsequent inverse Fourier transformation 
would yield a sharp image. For the sake of discussion let us assume that we have a 
reasonably good understanding of the blurring process and have a good approximation 
to psf and thus to the M.T.F. Most likely the blurring extends over a few spatial pixels 
and MTF will be small at high frequencies, i.e. large values of k. Blurring due to 
improper focusing, relative motion, atmospheric turbulence are in general of this type. 
The first two sources of blurring have MTF’s that vanish on contours in k-space. Where 
ever the MTF is small the contribution from noise, N/P, in equation 3b can be 
significant Consequently, approximating P by P/P can result in large noise 
contributions. The normal procedure to avoid this problem is to multiply P by a "filter" 
function which uses information about the noise to approximate 1/P where noise is 
unimportant and avoids contributions from regions where noise dominates. The Wiener 
filter is the most common filter discussed in the literature (2,3, 4, 6): 

(4) W' 1 - PH + Itf/F'Pl 2 ) 


This expression requires knowledge of the amplitude of the function to be calculated and 
the unknown noise contribution and cannot be used without further approximation. A 
simple method to avoid over emphasis of noise is present in section IV. 

n.3 Point-Spread-Function for Improper Focusing and Motion Blurring 


The point spread function describes how the intensity of any point of the "sharp" 
image is spread among neighboring points in the blurred image. In the case of improper 
focusing, the point spread function is angle independent and depends only on the 
distance from the initial point to neighboring points. 

In the simplest parameterization p is constant up to a given radius and zero for larger 
radii. An annulus of a given thickness could be added to allow the intensity to decrease 
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(5a) 


0* I*® = Pilfnl) 


smoothly to zero. 

In the case of linear blurring in the direction mg = [mg, n J, one must follow the 
path of a square pixel containing the original point in the sharp image. Since the pixel 
has finite dimensions pixels whose centers lie near but not necessarily on the line of 
blurring will receive contributions. In general these points will be located by a vector 
fn, which is the sum of a multiple of mg and a vector of at most unit length 
perpendicular to Iftg. Thus 


(5b) 


m 


$ 


I RBJ 


where q in the sum labels each such pixel center, and I, (>0) is its intensity, and 8^ is 
the Dirac delta function. 


HI Construction of the MTF from Blurred Images 


m.l Use of Zero Contours to Parameterize MTF’s 


In the most, common types of blurring, i.e. due to improper focusing and/or 
motion blurring, the Fourier transform of the point spread function, the so-called 
modulation-transfer-function MTF, has contours in the two dimensional frequency space 
along which it vanishes. In the cases where the blurring is uniform, i.e. the intensity in 
each square pixel is constant or zero, the MTFs for the cases of interest is the Airy 
function, J, (af)/af or the sinc-function sin (orf) / af, i.e. the diffraction patterns for a 
circular aperture and a slit Clearly these functions oscillate in sign and have their first 
zeroes at 1.22* and * respectively. Usually the FT of the blurred image, F 9 , allows visual 
identification of the nearest and perhaps other zero contours resulting from the blurring. 
In the case of motion blurring those parallel contours are more useful at determining the 
direction of blurring mg than the spatial image. 

m.2. Direct Computation of the MTF from the Blurred Image. 

If the blurred image has a straight edge or line, it can be used to determine the 
MTF directly without recourse to oversimplified assumptions about the psf. The basic 
idea results from considering the blurring of a feature in the sharp image. If the feature 
of interest were the only feature in the image, then equations 1 through 3 would still 
hold with f and F receiving contributions only from the feature. The MTF would of 
course not change. Consequently, if noise in the blurred feature can be neglected or 
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suppressed then the MTF would be the ratio of the FT of the blurred feature to the FT 
of the suppressed sharp feature. Edges and lines are especially appealing since they are 
geometrically simple. For simplicity, we will limit the discussion to them. In addition 
it should be realized that the profile of a blumed line can be trivially obtained from the 
profile of a blumed edge. 

In the following the edge or line of interest will be enclosed in rectangle whose 
sides are parallel or perpendicular to the supposed sharp line. Let vector m, = [m,,nj 
describe the direction and distance between pixels on the sharp line. Then the vectors 

T = [m,,nj and ~f= [-n,,mj 

are respectively parallel and perpendicular to the line. Each point (mjn) located by the 
vector from the origin m = [m,n] is simultaneously on a j-Iine which is parallel to the 
original line and characterized by 

j = mf = -mn, + nm, 

and or an i-line which is perpendicular to the original line and characterized by 


i = ml* = mm, + nn. 


It is easily seen that Ai =1 between neighboring perpendicular line s. Ai =1 between 
neighboring parallel lines, that the distance between lines is 1/k, = + n, 1 and the 

number of lines crossing between any adjacent points on a perpendicular line is k, 2 - 1. 

Neglecting end effects and assuming intensities are constant in the j -direction, 
then the intensity function for a sharp line, characterized by j, can be written as 

1W - 


and that of its blurred image as 

(6) lB<j) = Sp&*-% 

where j = J'-m and 

By replacing the sums in the FT over m' and n' by sums over i and j, one 
obtains the MTF 

(7a) Pi = l* {j) 

where the vectors nij are to a single point on each of the j-Iines. These vectors are to be 
selected such that the i sums in the ratio of L* and L* cancel. In general they are points 
lying closest to some line crossing the j-lines. In the case of improper focusing they can 
be taken to be closest to a particular i-line. In the case of motion blurring it would be 
those closest to a line parallel to 
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This problem is easily circumvented by considering the MTF on the line lc =xj 
(K=integer). Then 


(7b) 


*V = I’d) z^ 


To find the MTF for improper focusing one utilizes the fact that the appropriate 
MTF depends only on fkj and replaces k by |lc|4j| = (Tc|dc,; 


(7c) 


p£> = E (j) z (*10-4 v* 


To find the MTF for linear motion blurring one can utilize the fact that the 
appropriate MTF to first approximation varies only in the m B direction. Projection 
arguments easily then give 


(7d) 


pf> = £ i» (j) 


This can also be derived by writing the psf as 



then with y = 5^^ and 


t‘<n 


E 

4 


',\h, 


I 



Using m,,*m B , k, 2 k -fr^ <* (j-j,) k-^mj one obtains the expression for P a) . 

It should be realized that nothing can be learned about the MTF if the motion 
is parallel to the edge or line. This manifests itself in the m B 3 factor in the denominator 
of the exponent 


IV Restoration and Noise 
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All digitized images are subject to noise. The noise originates from the granular 
nature of the original photograph, from electronic noise and round-off error in the 
digitization process etc. Such noise can be both signal dependent and signal independent, 
spatially correlated and not, it can occur as "sparkle" or spikes in the F.T. of the image. 
Whatever the source or nature of noise, it complicates the deblurring or deconvolution 
process. This is especially true in the cases under consideration, improper focusing and 
motion-blurring where the MTF alternates in sign. Special care must be taken to avoid 
emphasis of noise in regions where the MTF changes sign, i.e. along the zero-contours. 

The standard approach to noise abatement is to use the MTF to construct a 
Wiener filter. Unfortunately such construction requires information on the noise that is 
not available. Also as pointed out by Castleman (2) such filters cannot handle image 
having large flat areas separated by sharp edges which is true of photographs of the 
shuttle. Consequently, a two-pronged approach to noise suppression was developed. The 
first involves using a two parameter high frequency component of the form 

S ' 1 = 1 + exp a (f - fj 

where f = |k| =i k 2 +? , f c is a variable cut-off frequency and a ' 1 = -{N/2 - i^A. The 
function S thus is vanishingly small at low frequencies, Vi at f = fe and approaches unity 
at high frequencies. 

The preliminary filter, Hp is then defined as 
Hp*‘ = P (1-S) + yS 

where y is the second variable parameter. Since the MTF is normalized to unity at zero 
frequency, H'' p , starts at unity follows the MTF until f = f t where it then follows 7 S and 
approaches yS and approaches y at high frequencies. The parameter y allows one to 
emphasize or de-emphasize the high frequency region relative to the low frequency 
region. 

This filter will clearly handle sparkle etc. noise at high frequencies, but it does 
not remove large values of Hp due to the vanishing of P at small and moderate 
frequencies. This is accomplished by introducing a third parameter, R, the maximum to 
minimum allowed values for the magnitude of H. Thus at each frequency point |Hp| 
is tested and replaced by R H,/|Hp| if it exceeds R.*This allows one to tune out or at 
least avoid over emphasis of noise near the zero contours of P. 

V. Results 


The main thrust of the research effort this summer was the designing and 
implementation of software to extract P from a blurred image. As a preliminary test of 
this software, it was applied to a Gaussian (5x5 pixel)-blurred image. Last summer’s 
work showed that the 5 x 5 pixel windows on the Gaussian was of consequence only 
at high frequencies. Thus the assumption that the psf was azimuthally symmetric would 
be reasonable since most of the signal is in the low frequence range. The resulting 
restored image was not quite as good as that obtained last summer where the 5 x 5 
window was taken into account, however the exercise clearly demonstrated that even in 
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the computer generated blurring information needed for constructing the MTF could be 
extracted from the blurred image. 

V.l. Deconvolution of An Improperly Focused Image 

Last summer an image was purposely blurred by defocusing the digitizing 
camera. Although there is no evidence in Figure la, the top 1/16 of the image is missing 
as a result of the digitization process. Such spatial-domaion truncation causes over 
shooting or ringing in subsequently filtered images (p.26 ref.l). Application of the newly 
developed software resulted in the MTF shown i Figure lb and gave a somewhat better 
restored image than that obtained last year. Last year the psf was approximated by a 
disc of constant amplitude (4 pixels in radius) and surrounded by a fuzzy annulus 2 
pixels in width (see Fig. 2a). As shown in Figure 2b, the psf derived from an edge in 
the blurred image (see Fig. la) is similar in size but has an intensity minimum in its 
center. Shown in Figure 3a, b are the Fourier transforms of the bluned and the restored 
image. 

As observed last summer, the Fourier transforms are much more sensitive to the 
restoration process than the spatial images. It is suggestive that fine tuning on the 
restoration process is much easier to observe on the Fourier image than on the image 
itself. The expected ringing is again present in the spatial image. The rectangular form 
of the Fourier transformed image is apparently due to the digitization process since it 
occurs in all images digitized with its video digitization system. 

V.2 Restoration of Two Images Bluned by Relative Motion 

Two images were blurred by moving them during the digitization process. The 
first was quite severely blurred in that the motion extended over about thirty pixels. The 
second was more moderate in that the blurring was over about a dozen pixels. In both 
cases the same edge circled in Fig. 5a was used in extracting the psf from the blurred 
image. Shown in Figure 3b,c, and 4b, c are the Fourier transforms of the blurred and the 
restored images. The original sharp image is essentially that shown in Figure la. In both 
cases the "zero contours" arc evident in the F.T. of the blurred image (see Fig. 3b and 
3c). These arc easy to recognize with the eye. The number of pixels to the first divided 
into N gives a good estimate of the blurring distance in the spacial domain. Similarly 
the Fourier transformed image can give a better determination of the direction of 
blurring than can be obtained from the spatial image. One simply uses PROFIL to define 
a line parallel to a maximum contour passing through k=0 and finds its slope 
parameters. Ak, Al, then m B :n B = Al:Ak, since the direction of blurring is perpendicular 
to extreme contours of P. This was used in Eq. 7d for determining P. 

Clearly the software can be used to restore images blurred by improper focus or 
by linear motion blurring if the blurred image contains a straight line or edge. 

VI Future Research 


In last summer’s report in addition to the program to construct MTF’s directly 
from blurred images there were two other areas of potential research. The first of these 
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involved the development of filters to reduce noise. Although some work has bene done 
along this line (see Section IV), many of the suggestions remain relevant. The use of 
windowing or image completion is important for incomplete images digitized by the 
Sony TV camera. However, these problems may be circumventted by using the Eikonix 
system. 

The main thrust of future work should be the development of a user-friendly- 
interactive-restorative system. The essential programs arc now in place, it is simply a 
matter of working them into a "cookbook" that can guide the inexperienced user to 
construct for any blurred image of interests the appropriate MTF and restoration filter. 
Such a program would use a decision-true format shown in Fig. 8. Photographs of the 
Space Shuttle Vehicle during launch and landing would contain straight edges and 
would be deblurred using an MTF extracted directly from the blurred images. 
Photographs taken by the Space Shuttle crews of agricultural, desert and ocean features 
would require parameterized MTF’s since they do not in general contain straight edges 
or lines. In such cases one would extract information needed to determine the parameters 
from their Fourier transforms. One would use a roster of the zeroes of the appropriate 
MTF, i.e. Airy or sine function. WARP would then be used to rotate the sine roster, and 
it stretch the roster until it overlaped the "zero" contours of F®. This would generate the 
appropriate parameters, e.g. the radius of the Airy disc or the length and direction of the 
motion blurring. 

The next step is to select the high frequency portion of the restoration filter. As 
presently envisioned this would have two parameters one a frequency, value which 
would be highest frequency where F* is free of "sparkle" noise, e.g. N/6 or N/4 
depending on the degree of blurring. The second parameter y which determines the 
relative emphasis of the high frequency region with respect to the very low frequency 
region, could be set equal to one or varied slightly to produce a restored F k where large 
amplitudes are mostly confined to the central region. 

The final step would be to select the maximum to minimum amplitude ratio, R, 
in the restoration filter. This would be done by taking a value like 10 and seeing if the 
"zero" contours persist or have been replaced by contours of large amplitudes in the 
restored F*. One would then raise or lower R until the amplutudcs in the regions of the 
contours are about the same size as their neighbors. 

After a restored F k is obtained which is similar to ones of less blurred images 
one would then take its inverse Fourier transform and obtain the restored image. Other 
features such as "sparkle" removal could be added later if they show promise of further 
improving the process. This should be a user-friendly-interactive program which guides 
the use along until a reasonable restored image is obtained. 
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Figure la, b 


Figure 2a, b 
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PHOTOGRAPHIC RESTORATION COOKBOOK 



Figure 5 







